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their understandings, these concept maps can be analyzed for representation 
of understanding from both quantitative and qualitative perspectives. In this 
study, a qualitative approach was utilized to extend the interpretations from 
the quantitative analysis of students’ conceptual understanding represented 
in their concept maps. A set of 22 seventh-grade maps was examined. Elements 
qualitatively examined included such components as reorganization in premaps 
and postmaps, changes in vocabulary usage, the nature of new knowledge 
representations, and the presence of misperceptions. Statistically 
significant differences may be noted in pooled preinstruction and 
postinstruction concept maps, but not in a class set of pre- and 
postinstruction maps, when a class set is analyzed qualitatively, changes in 
knowledge representations can be identified. Students were also asked to 
summarize in a written paragraph the understanding that their maps 
represented. The qualitative look at the maps can provide data to inform 
instruction in the content areas. Appendixes contain sample maps with a 
discussion of student support of maps and a set of student responses about 
the maps. (Contains 2 tables and 11 references.) (Author/SLD) 
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Students’ Conceptual Understanding: Qualitative Evidence in Concept Maps 



Abstract 

When students use concept maps to construct and represent their understandings, these 
concept maps can be analyzed for representation of understanding from both quantitative and 
qualitative perspectives. In this study, a qualitative approach was utilized to extend the 
interpretations from the quantitative analysis of students' conceptual understanding represented in 
their concept maps. Elements qualitatively examined included such components as reorganization 
in pre-maps and post-maps, changes in vocabulary usage, the nature of new knowledge 
representations, and the presence of misperceptions. Statistically significant differences may be 
noted in pooled pre-instruction and post-instruction concept maps, but not in a class set of pre- and 
post-instruction maps. When a class set is analyzed qualitatively, changes in knowledge 
representations can be identified. Students were also asked to summarize in a written paragraph 
the understanding that their maps represented. The qualitative look at the maps can provide data to 
inform instruction in the content areas. 



Introduction 

A challenge in education has been not only to help students elaborate conceptual 
understanding already possessed, but to modify those knowledge structures that contain 
misconceptions or alternative conceptions of frameworks (Novak & Gowan, 1984; Novak, 1990). 
To assist students in their development of conceptual understanding, concept mapping provides a 
two-dimensional, graphical representation for extemalization of concepts and relationships within a 
conceptual domain. Concept maps are based in large part on the work of Ausubel whose 
Assimilation Theory accords that learning becomes meaningful only in the context of the learner's 
prior knowledge (Aususbel, Novak, & Hanesian, 1978). 

Jonassen, Beissner, and Yacci (1993) note that through the process of developing a 
concept map the learner develops structural knowledge. Concept mapping requires that the learner 
take an active role in learning rather than being a passive recipient of knowledge, and students who 
have used concept mapping indicated that the act of mapping helped them understand the material, 
reduced memorization time, and helped clarify relationships among concepts (Heinz-Fry & Novak, 
1989). Concept maps can be instructional and learning tools that provide students with practice in 
relating concepts and can be implemented for individual construction or as a cooperative group 
activity. Utilization of concept maps for assessment purposes presents challenges for both scoring 
and interpretation, and the activity of concept mapping also requires instruction and practice to 
become "fluent" in the act of setting concepts out on paper or a computer platform. 

Methodology 

A pilot study to investigate the use of concept mapping as a learning and assessment tool 
was conducted in a set of rural, midwestem schools engaged in science education reform. A 
description of this study can be found in a project assessment report (Enger, 1996), and summary 
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statistics are available in the external project evaluation report (Burry-Stock, 1996). Eleven 
randomly selected teachers participated in training sessions that provided instruction in concept 
mapping and in assessment of student concept maps utilizing the Expert Science Teacher 
Educational Evaluation Model (ESTEEM) Concept Mapping Rubric (Burry-Stock, 1995). 

A set of 22 seventh grade maps on the respiratory system were examined to determine the 
evidence of learning that students provided in their maps and writing to support their maps. This 
kind of in depth look at maps seeks to address a question of the level of evidence of conceptual 
understanding represented in the students' concept maps. Reported in this paper is a qualitative 
examination of a set of maps to compare pre-instruction and post-instruction map organization, the 
nature of linkage vocabulary used, and evidence of students' misperceptions. Coupled with this is 
a sample students' written summaries of their knowledge representation on the post-instruction 
concept maps, and students' responses to the question taken from the ESTEEM Student Outcomes 
Questions (Burry-Stock, 1995), of what they thought their teacher wanted them to learn. Three 
samples of the student maps and their supporting paragraphs are found in Appendix A, and student 
responses as to what they thought their teacher wanted them to learn are provided in Appendix B. 

Discussion and Results 

The ESTEEM Concept Mapp in g Rubric addressed elements such as words indicative of 
concept understanding and meaningful connections (labeled lines) to show relationships among 
map words and map segments. As background for the overall total pattern of the maps was also 
judged, and for the scored maps, chi-squares were significant (p < .01) for the elements noted 
(Burry-Stock, 1996). Table 1 provides this data summary, and descriptive statistics for the total 
map scores are set out in Table 2. 



Insert table 1 about here 



Insert table 2 about here 



Burry-Stock notes that because the ESTEEM Concept Mapp in g Rubric evaluates levels of 
cognitive processing in the same manner across students from different classes, rubric use is 
appropriate with different content domains. The rater must, however, know the content to make 
the judgments necessary to evaluate the maps. 

The review of students' concept maps is time intensive and subjective when assessing 
maps with a rubric or for any qualitative evaluation. Misconceptions have been studied in many 
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areas in science, and the concept maps help to raise the instructor's awareness of these 
misconceptions in his/her students. The instructor can in turn can use this information to guide 
instruction and promote conceptual understanding. 

In reviewing the 22 seventh grade students' concept maps of the respiratory system, six 
students retained much of the same structural relationships on both pre- and post-instruction maps. 
Examples of maps that retained much of the original structure are included in Appendix A. Sixteen 
of the maps had more rearrangement in the post-map when pre- and post-instruction maps were 
compared, and an example of this is included in Appendix A. These sixteen students extended 
their concept maps and established more linkages. In this small sample of maps, the students 
whose maps tended to retain similar features in pre- and post- format were students who appeared 
to have greater conceptual understanding of the content that was studied about the respiratory 
system. This set of students may have been ready to move to more extensive or in depth 
information about the respiratory system. 

Students were asked to write a paragraph that summarized their post-instruction maps. The 
process of writing is intended to have the student elaborate their ideas and to reinforce 
understanding. The paragraphs can also be valuable in assessing misunderstandings that are 
represented on the maps. Student interviews have also been used to illicit student's conceptual 
understanding. 

When students use connecting lines to link words related to the concepts being studied, the 
word(s) written on those connecting lines should convey the nature of the relationship between the 
linked words. Without the linking words, the understanding of the relationship would need to be 
implied. However, such relationships are not necessarily implicit, and students have a tendency to 
omit linking words whether for lack of understanding or lack of attention to detail in writing. 

In examining the concept maps on the respiratory system, students conceptions that 
appeared to be inappropriate were that lung cancer is caused by bacteria, the lungs produce 
oxygen, that the lungs contract and expand, relax and contract, or inflate and deflate. From the 
lesson background provided by the teacher, it was indicated that the structure and function of the 
respiratory system were to be included in the instructional unit. Also stated in the teacher's 
instructional goals was the intent to have the students explain how the lungs act as an organ of 
excretion, to define respiration, and to understand the effects of disease on the respiratory system. 

Evidence from the maps suggests that the volume and pressure changes due to the 
movement of the rib cage, and the contraction or relaxation of the diaphragm would appear not to 
have been addressed. Understanding of the mechanics of inhalation and exhalation would seem to 
be conceptually appropriate in seventh grade science, and a recommendation might be made to 
include this in the science lessons for this unit of study. With concept mapping students should be 
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encouraged to indicate with words on their maps the relationship they intend. Why did they draw 
the line between or among the words on their maps? 

The students' responses to what they thought the teacher wanted them to learn can inform 
the teacher in aligning instructional goals and student outcomes. A question that may be posed 
with these students' responses would be that of whether or not student outcomes reflected the 
instructional goals and objectives. The students' maps appeared to have a focus that did not fully 
reflect the intent of the stated objectives, at least in the structure and function of the respiratory 
system. 

Summary Statements 

Clibum (1990) suggested that the instructor also should complete a concept map for the 
concept domain. This suggestion should be considered in that the instructor views where students' 
conceptual understanding is to be guided. Clibum further recommends that the concept map be an 
on-going activity to help build linkage and revisitation of concepts. Maps like any other visual aid 
capitalize on the extraordinary human capacity for visual memory. While concept mapping has 
limitations for large scale assessment (Ruiz-Primo & Shavelson, 1996), it can work very well as a 
classroom assessment and learning strategy. 

For successful learning, students need to take active possession of knowledge by seeking 
explicit connections between new concepts and those they already possess. The process of 
elaboration of personal meaningful knowledge occurs by restructuring existing conceptual 
frameworks (Regis, Albertazzi, & Roletto, 1996). Novak (1990) reported that a study conducted 
in junior high school classrooms showed that novel problem solving success was significantly 
correlated with success is concept mapping scores. Novak also noted that skill in concept mapping 
took at least one year to develop, and development of this skill showed little correlation with ability 
and achievement test scores. Concept mapping when coupled with other educational strategies led 
to superior achievement It is for these reasons that concept mapping can be a viable way to help 
students build conceptual understanding. 
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Table 1 

Comparison of Pre- and Pos t-Instruction Concept Maps 



Scoring Item 


df 




1. Key words 


16 


108.37** 


2. Non-key words 


25 


202.99** 


3. Connecting lines 


12 


6.90 


4. Connecting lines labeled 


25 


100.29** 


5. Relationships between concepts 


25 


64.60** 


6. Relationships between segments 


25 


87.96** 


7. Hierarchical pattern 


25 


53.79** 


8. Deep understanding 


25 


57.15** 



n = 175, **p < .01 
Table 2 

Descriptive Statistics for Pre- and Post-Instruction Concept Maos 



Total Map Scores 


Mean 


Standard Deviation 


t 


Pre-instruction maps 


20.39 


7.93 


- 7.99** 


Post-instruction maps 


26.99 


8.09 




n = 175, **p < .01 
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Appendix A 

Sample Pre-Instruction Map: Student 8 
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Appendix A 

Sample Post-Instruction Map: Student 8 
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Appendix A 
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Appendix A 

Sample Post-Instruction Map: Student 12 
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Appendix A 

Sample Pre- Instruction and Post- Instruction Maps: Student 25 
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Student 8 



Appendix A 

Student Support of Maps 



What the teacher wanted us to learn 

To learn how the parts of the respiratory system are built and how each part works. Learn 
how the respiratory and excretory systems work together. Learn how aU body systems are 
connected. To learn how to act as a scientist, and learn the processes of science for 
investigations. 



Student summary of post-instruction map 

When you inhale (breath in) you are taking in oxygen, when you exhale (breath out) you 
are getting rid of carbon dioxide. Many things are bad for your lungs and respiratory 
system. One is smoking and another is drugs. Smoking can cause diseases such as 
cancer. Germs can cause illnesses in your throat also - it is important you protect your 
lungs and respiratory system so you can stay healthy. 

Student 12 



What the teacher wanted us to learn 

Use the processes of science in experements. Understand how the respiratory system is a 
part of the excretory system. Understand how systems of the body relate to one another. 
Understand the structure and function of the respiratory system. 

Student summary of post-instruction map 

The respiratory system can be divided into three different groups that are diseases, 
breathing, and the funtion. Breathing starts in a tube called the trachea. In the trachea you 
inhale oxygen and exhale carbon dioxyde. Nerves in your brain effect how you breathe. 
The function of the respiratory system starts with the nose, mouth, lungs, and diaphragm. 
The lungs have bronchi which branches out into bronchial tubes. At the end of the 
bronchial tubes are aveolies. The functions of the diaphragm is to expand and relax. The 
diaphragm is a smooth muscle. Germs cause many diseases. Some diseases are asthma, 
phnamonia, and the flue. In conclusion many things effect the way people breath. 

Student 25 



What the teacher wanted us to learn 

I think the main ideas were to understand the structure and function of the respiratory 
system, to know how it is a part of the excretory system, know how the systems of the 
body relate to each other, and to be able to use this processes of science into investigations. 

Student summary of post-instruction mao 

Respiration has ten key words. They are diseases, lungs, oxygen, exhale, inhale, trachea, 
ribs, carbon dioxide, breathing, and diaphragm. I found twelve non-key words. I found 5 
linking words. So in conclusion, there are a lot of non-key words and linking words that I 
missed putting on my concept map. 
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ESTEEM Rubric Concept Map Scores 


Appendix A 
Student Map Scores 




Student 


Pre-Instruction 


Post-instruction 


Student 8 


25/40 (63%) 


25/40 (63%) 


Student 12 


32/40 (80%) 


36/40 (90%) 


Student 25 


16/40 (40%) 


25/40 (63%) 
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Appendix B 
Student Responses 

Want do you think your teacher wanted you to learn during this unit of study? 

1 To understand the structure and the functions of the respiratory system. To understand 
how the respiratory system is part of the excretory system. To understand how all of the 
systems in the body relate to one another. 

2 He wanted us to learn the structures and functions of the respiratory system, how the 
respiratory system is part of the exeretory system, the processes of science in 
investigations, and systems of the body relate to each other. 

3 (Not Available) 

4 How the respiratory system works and the structure. How all the systems of the bodies are 
linked together. How to investigate the systems. 

5 We were to understand the significant structure and function in the respiratory system. 
Respiratory and excretory - together - systems. Investigations. 

6 To learn how to exercise properly and what your heart-rate had to be before it did you any 
good. To understand how respiratory and excretory are related. How to diagram heart 
rates. Learn to do averages and process of elimination and processes of science. 

7 Put the processes of science into effect. Understand how the respiratory and excretory 
systems relate. Understand how other systems relate to each other. Understand how the 
respiratory system works and how it's built. 

8 To learn how the parts of the respiratory system are built and how each part works. Learn 
how the respiratory and excretory systems work together. Learn how all body systems are 
connected. To learn how to act as a scientist, and learn the processes of science for 
investigations. 

9 Structure and function of the respiratory system. How body system relates to each other. 
Process of science investigations. How respiratory system is a part of the excretory 
system. 

10 Our teacher wanted us to learn the structure and function of the respiripratory system, 
which includes the lungs, trachea, wind pipe, nostril, diaphram, ect. Our teacher also 
wanted us to learn how the respiratory system is a part of the excretory system is because 
without breathing the chemicals could not move in the blood stream because it wouldn't be 
working properly. He also wanted us to know how systems of the b ody relate to each 
other in breathing and dexocrine systems and other parts. He also wanted us too know 
how to use the processes of science in investigating which he’s done well because we 
know now how to do this. 

1 1 (Not Available) 

12 Use the processes of science in experements. Understand how the respiratory system is a 
part of the excretory system. Understand how systems of the body relate to one another. 
Understand the structure and function of the respiratory system. 
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13 Learning how respiratory is related to the excretory system. Learning about structure and 
function. Learn how system and body work together to help you breath. Using scientific 
processes in investigations of respiratory system. 

14 Understand the respritory system structure, function. Learn how the resprtory system is 
related to the excretory. Use sceitific process in problems. 

15 To get in our heads the function and structure of the respiratory system. To see that the 
respiratory system is a part of the excretory system. To realise how all of the systems of 
the bodys relate to each other. And to use and realixe what the process of science is in a 
investigation. 

16 (Not Available) 

17 We breathe with our diaphragm. Breathing is a complicated process. Respiration is also 
excretory. 

18 Know how the resportory system fuctioned and the struture. How other systems were 
related to each other. How science was used in investigations. 

19 To be able to identify the structure and function of the respiratory system. To tell how the 
respiratory system is part of the excretory system. Tell how systems of the body relate to 
one another. 

20 Understand respiratory's structure and function. Respiratory system is part of the 
excreatory system. How body systems relate. In investigations use the processes of 
science. 

21 How the systems of the body relate to each other. How the respiratory system works. 

Use the science process in our investigation. 

22 I think he wanted us to learn the many different parts of the respiratory system. Another 
main idea is that he wanted us to learn how the different body systems work together, like 
the respiratory and exeratory system work together. Another main idea of this unit was to 
use the processes of science in our experiments/investigations. 

23 I think my teacher wanted me to learn and understand about structure and function of the 
respitory system, so I know more about later on. Plus to know how the different systems 
relate to each other. 

24 How systems relate to each other. To use science in investigations. How different 
systems work together. 

25 I think the main ideas were to understand the structure and function of the respiratory 
system, to know how it is a part of the excretory system, know how the systems of the 
body relate to each other, and to be able to use this processes of science into investigations. 
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